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STUDENT POINTS OP VIEW IN RATINGS OF COLLEGE INSTRUCTION 



Abstract 

This study Investigated student points of view In their ratings of 
specific courses and Instructors by analyzing student responses within each 
of three classes separately, and then sought to generalize the results furth 
by additional analyses with 300 students randomly selected from 402 classes 
in five colleges. Different points of view were found for student ratings 
of course examinations, textbooks and supplementary readings, and class 
discussions. These various points of view were moderately related to such 
student characteristics as grades and sex, although not in all three of the 
classes studied. This last point underscores the importance of the context 
(i.e., the particular course) in understanding or interpreting the meaning 
of student ratings. 



STUDENT POINTS OF VIEW IN RATINGS OF COLLEGE INSTRUCTION 



John A. Centra Robert L* Linn 

Educational Testing Service University of Illinois 

Most people who have taught realize that frequently there arc several 
types or subgroups of students In the typical class, each of which may be 
reacting a little differently to the teacher and the course. These different 
reactions might be expected to be reflected in the end-of-course student 
ratings given the Instructor, If so, there should be some way of identifying 
and describing these subgroups of students so that a better understanding of 
this diversity of viewpoints might be gained, This study, then, focuses on 
the different so-called ''points of view" that students may have in their 
ratings of their college teachers and courses, 

Rees (1969) also attempted to determine factors which represeiit points 
of view in student ratings of teachers. Students in the Rees study rated 11 
types of teachers representing seven academic areas (e,g«, English teachers, 
music teachers, history teachers). The resulting points of view, therefore, 
did not reflect differences among students in their reactions to specific 
instructor behavior, but rather students' notions about types of teachers 
in different fields <)f study. 

By contrast, Yjnge and Sassenrath (1968) Investigated the relation^ 
^|hlp be|:ween student ratings .of instructors and the personality ratings for 
each of three sjSeciflc Instructors, Their results indicated that while 
personali\:y scores were related to student ratings of instruction, the find- 
ings were not consistent for all three instructors. That is, the type 
of student who tended to rate one Instructor high on a given factor might 



\ 

very well be the type of student who would rate another Instructor low on 
the same quality. Yonge and Sassenrath concluded by emphasizing the Itnportance 
of context — that is the particular course in which ratingo are collected — as 
well as student points of view in interpreting student ratings of instruction. 

The present study further investigates student points of view of 
specific instructors and courses by analyzing student responses within 
each of three classes separately and then, in a second study, analyzing 
responses across a much larger sample of courses. After identifying student 
points of view based on responses to the rating items, varloas student charac- 
teristics variables are related to the points of view to determine the extent 
to which these variables differentiate groups of students. 

Method 

As indicated two separate studies or analyses were conducted. The 
first study involved an obverse (or inverse) factor analysis within each of 
three large classes taught at a major university. This analysis resulted in 
the identification of several subgroups of students which were further analyzed 
by means of discriminant analysis. The second study sought to generalize the 
results further by additional analyses with 300 students randomly selected 
from 402 classes in five colleges. The instruments used in both studies 
were two highly similar versions of the Student Instructional Report 
(Centra, 1972). 
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study 1 

A natural science > a social science and a humanities class taught at 
a large mldwestem university were used for the Initial study. Morft 
specifically the thi'ee classes and the number of students selected randomly 
from each were: General Physics (103), Social Inequality (119), and Man and 
Religion (67). 

Students In each of these classes had responded to the 1971 version of 
the Student Instructional Report (SIR)» The Instrument Included some 29 
items eliciting student ratings in such areas as the course organization 
and procedures, faculty-student interaction, lectures, assignments, and 
examinations, The items were scored on a one to four or one to five scale, 
with the higher score more favorable, Also included were five items dealing 
with individual student characteristics such as grade point average and 
expected grade in the course. 

An obverse factor analysis using the 29 rating Items to identify groups 
of students within each of the three classes was performed. Instead of 
analyzing the item or variable correlation matrix, which is usually the 
case, an obverse factor analysis analyzes the subject correlation matrix 
in an attempt to identify groups of individuals with similar patterns of 
responses. For each of the three courses, a principal axis analysis of the 
student correlation matrix was employed . 

Results 

On the basis of the break in the magnitude of the latent roots following 
the fifth root, five factors were rotated, A varlmax rotation to five factors 
resulted in five groups of students for each course. Students with loadings 
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of ,40 or higher were then noted, resulting In the following number of 
students within the five groups in each course: 

General Physics; 30, 23, 23, 20, 6 

Social Inequality: 45, 35, 16, 14, 6 

Man and Religion: 19, 14, 11, 16, 5 

These figures total slightly less than all students in the classes 
because a few students did not have loadings of at least .40 on any factor. 
The five or six students in the fifth group for each class would appear to 
be a minimum number to be considered a separate subgroup. To investigate 
how the groups differed in their responses to the SIR items, a discriminant 
analysis using the five groups and the 29 rating IteM was run for each of 
the three classes. Four discriminant functions for each course were 
significant (p < .05), but this was to be expected since the groups had been 
selected on the basis of how they differed in item responses. Of primary 
interest from the discirlmlnant analysis were the items that had high corre- 
lations with the four discriminant functions. These results are presented 
In Table 1, Correlations between the four discriminant functions and the 
items are reported in Table 1 when the correlations are ,20 or higher in 
absolute value. This is repeated in Table 1 for each of the three classes* 

Insert Table 1 about here 

Especially important for identifying how the groups differed in their 
ratings of instruction are the items that correlated fairly high with one 
or more of the discriminant functions, particularly if this occurred In 
two or three of the classes. Items in this category include: 



- Whether the Instructor made helpful comments on papers or exams* (Item 12) 

- The overall value of class discussions. (Item 24) 

the overall effectiveness bf the instructor. (Item 25) 

- Overall rating of exams. (Item 22) 

- Overall rating of supplementary readings. (Item 21) 



and 




vniether the catalog description adequately describes the content 
and method of the course* (Item 17) 

Moderate correlatlops may be noted for such items as the extent to 
which students thought they had been challenged by the course (Item 19), 
whether lectures were too repetitive of material In the textbook (Item 8), 
and whether the institictor was readily available to students (Item 6). 

While the preceding analysis helped describe how the groups differed 
on the student rating items, a second question was whether known individual 
student characteristics discriminated among the groups. That Is, could 
the groups be differentiated according to certain student information about 
the student? The student characteristics information, obtained ftom students 
in the last part of the questionnaire, included the following items: (1) 
major field (scored 1 if the course belonged to the student* s major and 0 
otherwise); (2) expected grade in the course; (3) cumulative grade point 
average (GPA) ; (4) level (freshman « 1, sophomore « 2, junior « 3, senior ^ 4| 
and graduate « 5); and (5) sex (female « 1, male = 2). 

To Investigate this question a second discriminant analysis, this time 
with the five groups within each class and the five student characteristics : ■ 
variables, was run next for each course. The results, summarized In Table 2| 
Indicate that in only the Social Inequality course were any of the discriminant 



functions significant. In that course the first function was significant 
(p < .007). 

Insert Table 2 about here 

The correlations of this function with the five student characteristics 
variables, reported in Table 3, ii»dicated that student class level and grade 
expected in the course were most highly correlated, followed by cumulative 
grade point average. Students in groups that were high on the function 
tended to be freshmen and sophomores, to expect high grades in the course, 
and to have higher cumulative grade point averages* Groups at the other end 
of the continuum tended to consist of upperclassmen with lower expected 
grades in the course and a relatively lower GPA. The group that was 
highest on the student characteristic function was also lowest on the second 
function of the instructional rating items (coltimn six of Table 1). This 
latter function had its highest correlations on the rating of exams and class 
discussions, as well as whether students thought the instructor made helpful 
comments on papers or exams. Thus for the Social Inequality course, lower- 
classmen who expected a good grade in the course and who had a relatively 
high GPA tended to rate exams and class discussions high, and also agreed 
that the instructor's cotnments on papers or exams were helpful. 



Insert Table 3 about hete 



Study 2 

The preceding analyses In Study 1 revealed Identifiable subgroups or 
points of view within each of three classes. But what kinds of subgroups of 
students would be found within a larger sample of classes and how might 
the groups be distinguishable? To answer these questions a random sample 
of 300 students was selected from 402 classes at five colleges* A 1972 
version of the Student Instructional Report, which contained 31 items 
rating instruction as well as the five student characteristics items, was 
used for this study* As mentioned earlier, the items in this form overlapped 
considerably with those in the 1971 version of SIR used in Study 1* The 
scoring syst-c^ai for the two forms was identical* 

Because differences in student responses would reflect differences 
between classes as well as individual differences, the first step in the 
analysis was to subtract the class mean from each student response. The 
resulting 300 x 31 matrix, then, consisted of deviation scores — specifically 
the difference between each student* s response to each item and the mean 
response of the class* The values thereby obtained were dependent on 
differences among individuals within classes and included effects common 
to all classes. The second step followed a decomposition procedure developed 
by Eckart and Young (1936), and discussed by Tucker and Messick (1963), 
for uncovering the variety of points of view (or individual perceptual 
structures) represented in a total group* The Eckart-Young decomposition 
procedure provides two matrices: one of them characterizes the within ^ ^^^^^^^^^^^i^ ; 
subjects (students) variation and the other characterizes the within Items 
variation. By inspecting the eigenvalues of the decomposition, It was 



decided that three dimensions were adequate to describe the subject and Item 
space. The first six eigenvalues were: 36.52, 18.12, 16.83, 14.56, 13.91, 
and 13.44. An oblique rotation of the subject space was next performed 
and the same transformation (inverted) was applied to the item space (see 
Pennell, 1972, equation 3). 

In order to Investigate the relationship of student characteristics to 
the dimensions that were identified, the subject scores on the three rotated 
dimensions were correlated with the five student characteristics items (majo 
expected grade, cumulative GPA, class level, and sex). 

Results 

The loadings on the three dimensions of the matrix that characterizes 
the within item variation are presented in Table 4. Factor I has its 
highest loadings on "Rating of quality of exams" (Item 34), "Inst%uctor 
knew when students didn't understand (the materiaiy (Item 5), and "Exams 
reflected important aspects of the course" (Item 17). This might be called 
an Examinations factor. 



Insert Table 4 about here 
-t - _ — 



Factor II is primarily defined by "Textbook rating" and "Supplementary 
readings rating." For want of a better name this is labelled the Text 
factor. The third factor is called Course Quality and has Its highest 
ioidings on "Value of the course to the student" (Item 38), "Rating of 1 
the quality of the discussions" (Item 36), "Rating of the quality of the , 
lectures" (Item 35), and "Work load of the course" (Item 22). 
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The correlations of the factor scores with the five Individual char- 
acteristics are reported In Table 5i Factor I correlates between .37 
and #43 In absolute value with all five of the individual characteristics to 
Higher ratings of the Examinations factor are more apt to be received from majors 
than from nonmajors, from students with high expected grade than low 
expected grade, from students with high GPA than low GPA, from lowerclassmen 
than upper classman, and from women than men* A similar pattern was also 
found for Factor HI, Course Quality^ except that the range of correlations 
was somewhat greater. They ranged from a low in absolute value of .26 
for major to a high In absolute value of .48 for level. Thus, the Judgment 
of Course Quality varies substantially with student class level, somewhat 
less with expected grade, GPA, and sex, and least with maJor» 

Insert Table 5 about here 

The correlations between Factor II> Text , and the five individual 
characteristics were much lower in level but the pattern of the signs of 
the correlations was the same as that for Factors I and III, The upper- 
classmen tended to be slightly more critical of the texts than the lower- 
classmen (r -.21). The other significant correlation (p < .05) was the 
.18 between Factor II and major. Thus, on the average, majors viewed the 
textbooks for the course slightly more favorably than did the nonmajors. 

'i:£)-MMM^^^^ Discussion 

Both of the studies reported here indicate identifiable points of 
view in student ratings of instruction. These points of view, represented 

O 

r 
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by factors, totalled as many as five for each of the larger classes included 
In this study, but would probably number fewer for smaller classes. They 
were distinguishable by the pattern of responses made to the instnictional 
rating Items. For example, analyses within each of three classes separately 
(Study 1) and across some AOO classes (Study 2) both indicated that groups 
of students differ in their ratings of the course examinations, In the text- 
book and supplementary readings, and in class discussions. In addition, 
students differed in their viewpoints regarding the value or challenge of 

the course to them. 

All of these features— examinations, assignments, class discussions- 
would appear to be aspects of a course for which different types of students 
might understandably differ in their reactions. On the other hand, there 
were minimal differences on such aspects as the extent to which a course is 
organized^ and the extent to which the instructor is available for help. 

To what degree were thpae various points of view associated with 
Identifiable student characteristics? For the five student variables Included 
In the questionnaire, there was a moderate relationship. In Study 2, majors 
(vs. nonmajors), students with higher expected grades and high cumulative 
grade point averages, lowerclassmen (vs. upperclassmen) and/Women (vs. men) 
tended to rate higher the Exami nations . Course Quality and, to a lesser 
extent, the Text factors. The tendency for nonmajors in a field, or students 
with lower grades (or expected grades), to rate a course lower in these 
particular aspects is not surprising; students With less of an Interest in 
a course might be expected to be more critical of examinations or course 
worth, as would students whose academic performance has been less than out- 
statding. The moderate relationship would, however, suggest that for some 
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courses these particular student variables do not distinguish the points 
of view. This In fact Is what the results of Study 1 suggest* In only 
one of the three courses did student characteristics discriminate aioong the 
points of view. And for this one course the results paralleled the results 
of Study 25 lowercld3smeni those who expected a high grade and those with 
a relatively high GPA, tended to rate exams and class discussions higher. 
For two of the three courses, however, none of the dlscrlmittdnt functions 
were significant for the five groups which represented different student 
points of view. 

It should be pointed out that there are any number of student char- 
acteristics variables that might Identify points of view of Instruction^ 
Student need for iicademlc achievement or social approval are just two that 
might affect their ratings of examinations or class discussions. And, 
as the results of this study sUggesti somi? of these student characteristics 
would not necessarily affect the ratings of very many courses or instructors 
in exactly the same way* This underscores the point made by Yonge and 
Sassenrath (1968) that the context of the ratings (i.e*, the particular 
course and instructor) are of t^n important in understanding or Interpreting 
the meaning of student ratings. 

What, then, ate the implications of this study for faculty members 
who use student ratings? It would appear that teachers should look at the 
distribution of student responses to each rating item, and not only at the 
average response for the class. This would be particularly important for 
Items that deal with class examinations, discussions^ and assignments. While 
it may not be possible, or som6 might argue even desirable, for teachers to adjust ' 
all aspects of a course to student expectations ^tefiich^rs ought to at least be 
O of how they Interactihg xJTtW^^I^^ segments bf^tlke 'cUS§. 
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Table 2 

Summary of Discriminant Analysis for Each Class and 
the Five Student Characteristics Variables 





df hypothesis 


df 


F 


P 


Physics 


20 


309 •A 


.92 


.564 


Social It.aquallty 


20 


355.8 


2.00 


.007 


Man and His Religion 


20 


186.7 


.79 


.729 



-15- 



Tabie 3 

Correlations between the Student Characteristics Variables and 
the Discriminant Function for the Social Inequality Course 



Student Variables Correlations 



Major Field vs. 


Other 


-.13 


Cumulative GPA 




.43 


Grade Expected 


in Course 


.51 


Class Level 




-.66 


Sex 




.23 



Table 4 

Dimensions of the Item Factor Matrix 



Loadings on ^ 

item Each Dimension "'^ 

I II III 

Examinations Text Course Quality 



1 . 


Instructor objectives have been made clear 


.66 


-.24 


.86 


2. 


Agreemant between objectives and vhat was taught 


.72 


-.21. 


.72 




Instructor used class time well 


.68 


-.60 


.91 


4 . 


Instructor was readily available to students 


.78 


-.35 


.23 




Instructor knew when students didn't understand 


1.46 




.29 


0. 


Lectures too repetitive of textbook 


.39 


* 11 


-.90 


7 • 


Instructor encouraged students to think 


.67 


-.21 


• 57 


8, 


Instructor seemed concerned with students* progress 


1.25 


-.21 


• 63 


^* 


Instructor made comments on papers or exams 


1 .13 


^•12 


• 25 


10. 


Instructor raised challenging questions 


.52 


-.66 


A i 

.94 


11. 


Student felt free to ask questions 


1.09 


-.56 


.41 


12. 


Instructor was well-prepared for each class 


.70 


-.22 


.79 


13. 


Instructor told students how they would be evaluated 


.91 


-.09 


.47 


14. 


Instructor summarized major points In lectures/disc , 


.75 


-.27 


.61 


15. 


Student Interest In subject has been stimulated 


1.08 


-.03 


1.15 


16. 


Scope of course has been too limited 


.30 


-.07 


-1.12 


1?. 


Exams reflected important aspects of the course 


1 M 


.27 


-.04 


18. 


Student effort in the course 


-.17 


.18 


1.32 


19. 


Instructor openness to other viewpoints 


.96 


-.56 


.46 


20. 


Instructor has accomplished ^course objectives 


1.15 


-.39 


.86 


21* 


Level of difficulty of the course 


1.07 


.17 


-1.11 


22, 


Work load of the course 


.91 


.14 


-1.51 


23, 


Pace of the course 


.77 


.04 


-.66 


24. 


Use of examples or illustrations 


.32 


.17 


.37 


32. 


Textbook rating 


.59 


2.78 


1.16 


33. 


Supplementary readings rating 


.74 


2.60 


1.01 


34. 


Rating of quality of exams 


2.01 : 


1.04 


♦ 64 


35. 


Rating of quality of lectures 


1.22 


.02 


l.Sl 


36. 


Rating of quality of discussions 


1.17 


.11 


1.63 


38, 


Value of the course to student 


1.23 


.68 


1.67 


39. 


Overall effectiveness of Instructor 


1.22 


.17 


1.26 



:]':t::\'^^int^ the original score matrix rather than a correlation matrix is being 
|iE>|i|:f5k^ are to b6 expected. 



-17- 



Table 5 

Correlations between Factor Scores and 
Five Individual Characteristics 







Factor 




Individual 
Characteristic 


I 

Examinations 


II 
Text 


III 

Course Quality 


Major 


.37 


.18 


.26 


Expected Grade in Course 


.AO 


.06 


.44 


Cumulative CPA 


.40 


.07 


.42 


Level 


-.42 


-.21 


-.48 


Sex (F=1,M=2) 


-.43 


-.10 


-.40 



